Imaging apparatus and controlling method

ABSTRACT

An imaging apparatus according to embodiments of the present disclosure includes a sensor unit, a front engine that generates compressed raw image data by processing image data acquired from the sensor unit, a main engine that executes a development process on the compressed raw image data acquired from the front engine, and a display unit that displays an image. The front engine controls the display unit to display an image based on the image data acquired from the sensor unit, and the main engine records the image data subjected to the development process in a recording medium.

BACKGROUND Field of the Disclosure

The present disclosure relates to an imaging apparatus and an imagingmethod.

Description of the Related Art

Japanese Patent Application Laid-Open No. 2013-197608 discusses animaging apparatus that uses a front engine and a back engine to processimage data acquired by an imaging circuit.

The imaging apparatus discussed in Japanese Patent Application Laid-OpenNo. 2013-197608 causes the front engine and the back engine to operatewhile a live view image is displayed. Thus, even while the live viewimage is displayed, power to drive the two engines is required.

SUMMARY

According to embodiments of the present disclosure, an imaging apparatusincludes an imaging sensor, a display unit configured to display animage, a first processing circuit configured to control the display unitto display an image based on image data acquired from the imagingsensor, a second processing circuit configured to record in a recordingmedium the image data acquired from the imaging sensor via the firstprocessing circuit, and a control unit configured to control supply ofpower to the second processing circuit, wherein the control unitrestricts the supply of power to the second processing circuit in animage capturing standby state, and, upon input of an instructionregarding an image capturing operation for recording the image dataacquired from the imaging sensor in the recording medium, lifts therestriction of the supply of power to the second processing circuit.

Further features of the present disclosure will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are external views of a digital camera.

FIG. 2 is a block diagram illustrating an example of a configuration ofthe digital camera.

FIG. 3 is a block diagram illustrating a data flow in an image capturingmode.

FIG. 4 is a block diagram illustrating a data flow in a reproductionmode.

FIGS. 5A and 5B are flowcharts illustrating an operation control flow ofthe digital camera.

FIG. 6 is a schematic diagram illustrating an example of a menu screenfor setting an external output function.

FIG. 7 is a flowchart illustrating a flow of a live view process.

FIG. 8 is a flowchart illustrating a flow of an imagecapturing/recording process.

FIG. 9 is a flowchart illustrating a flow of a reproduction process.

FIG. 10 is a flowchart illustrating a flow of a live view outputprocess.

FIG. 11 is a flowchart illustrating a flow of a reproduction outputprocess.

FIGS. 12A and 12B are flowcharts respectively illustrating flows of astart process for starting a main engine and a stop process for stoppingthe main engine.

FIG. 13 is a diagram illustrating an example of color filter arranged ina sensor unit.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present disclosure will be described indetail below with reference to the attached drawings.

FIGS. 1A and 1B illustrate external views of a digital camera 100 as anexample of an apparatus according to an exemplary embodiment to whichthe present disclosure is applicable. FIG. 1A is a front perspectiveview of the digital camera 100. FIG. 1B is a rear perspective view ofthe digital camera 100.

A display unit 101 is a display unit that is provided on the backsurface of the digital camera 100 to display an image and various piecesof information. In the display unit 101, a touch panel 111 capable ofdetecting a touch operation on a display surface (an operation surface)of the display unit 101 is provided. The touch panel 111 is included inan operation unit 110.

A shutter button 102 is an operation member for giving an imagecapturing instruction.

A power switch 103 is an operation member for switching turning on andoff of the digital camera 100. The power switch 103 inputs a startinstruction to turn on the digital camera 100. The power switch 103 alsoinputs a stop instruction to turn off the digital camera 100.

The operation unit 110 represents some or all of operation membersincluded in the digital camera 100. The operation unit 110 includes thetouch panel 111, a main electronic dial 112, a sub electronic dial 113,a cross key 114, and a SET button 115. The operation unit 110 alsoincludes a moving image button 116, an automatic exposure (AE) lockbutton 117, an enlargement button 118, a reproduction button 119, a menubutton 120, and a mode switch 121. The operation unit 110 may includethe shutter button 102 and the power switch 103. The functions of theoperation members included in the operation unit 110 may be assigned tobuttons physically different from each other, or a plurality offunctions can also be executed using a single button. In the presentexemplary embodiment, for illustrative purposes, the shutter button 102and the power switch 103 are treated as operation members separate fromthe operation unit 110.

The main electronic dial 112 is a rotary operation member included inthe operation unit 110. By rotating the main electronic dial 112, it ispossible to change the setting value of the shutter speed or thediaphragm. The sub electronic dial 113 is a rotary operation memberincluded in the operation unit 110 and enables the movement of aselection frame or image advancement. The cross key 114 is a cross key(four-direction key) included in the operation unit 110. Upper, lower,left, and right portions of the cross key 114 can be pushed. The crosskey 114 enables an operation corresponding to a pushed portion of thecross key 114. The SET button 115 is a push button included in theoperation unit 110 and is mainly used to determine a selection item. Themoving image button 116 is used to give an instruction to start or stopthe capturing (recording) of a moving image. The AE lock button 117 isincluded in the operation unit 110. By pressing the AE lock button 117in an image capturing standby state, it is possible to fix an exposurestate. The enlargement button 118 is an operation button included in theoperation unit 110 and used to turn on and off an enlargement mode inlive view (LV) display in an image capturing mode. By turning on theenlargement mode and then operating the main electronic dial 112, it ispossible to enlarge and reduce an LV image. In a reproduction mode, theenlargement button 118 functions as an enlargement button for enlarginga reproduction image and increasing the enlargement ratio. Thereproduction button 119 is an operation button included in the operationunit 110 and used to switch the image capturing mode and thereproduction mode. The reproduction button 119 is pressed in the imagecapturing mode, whereby the image capturing mode transitions to thereproduction mode, and the latest image among images recorded in arecording medium 200 can be displayed on the display unit 101. The menubutton 120 is included in the operation unit 110. If the menu button 120is pressed, a menu screen where various settings can be made isdisplayed on the display unit 101. A user can intuitively make varioussettings using the menu screen displayed on the display unit 101, thecross key 114, and the SET button 115. The mode switch 121 is anoperation member for switching various modes.

A communication terminal 104 is a communication terminal for the digitalcamera 100 communicating with a lens attachable to and detachable fromthe digital camera 100 or with an adapter to which the lens isconnected. An eyepiece portion 11 is an eyepiece portion of an eyepieceviewfinder (look-in type viewfinder). The user can visually confirm avideo displayed on an electronic viewfinder (EVF) 108 within theeyepiece viewfinder through the eyepiece portion 11. An eye approachdetection unit 12 is an eye approach detection sensor that detectswhether the eye of a user approaches the eyepiece portion 11.

A terminal cover 13 is a cover for protecting a terminal (notillustrated) for connecting the digital camera 100 to an externaldevice.

A cover 14 is a cover of a slot in which the recording medium 200 isstored. A grip portion 15 is a holding portion shaped so as to be easilygripped with the right hand when the user holds the digital camera 100.The shutter button 102 and the main electronic dial 112 are placed atthe positions where the shutter button 102 and the main electronic dial112 can be operated with the right index finger in the state where thedigital camera 100 is held by gripping the grip portion 15 with theright little, third, and middle fingers. The sub electronic dial 113 isplaced at the position where the sub electronic dial 113 can be operatedwith the right thumb in the same state.

FIG. 2 is a block diagram illustrating an example of configuration ofthe digital camera 100 according to the present exemplary embodiment. InFIG. 2, a lens unit 150 is a lens unit including an interchangeableimaging lens. Although a lens 151 normally includes a plurality oflenses, FIG. 2 illustrates only a single lens in a simplified manner. Acommunication terminal 153 is a communication terminal for the lens unit150 to communicate with the digital camera 100. The communicationterminal 104 is a communication terminal for the digital camera 100 tocommunicate with the lens unit 150. The lens unit 150 communicates witha system control unit 132 via the communication terminals 153 and 104and causes a lens system control circuit 154 within the lens unit 150 tocontrol a diaphragm 152 via a diaphragm driving circuit 155. The lensunit 150 communicates with the system control unit 132 via thecommunication terminals 153 and 104 and causes the lens system controlcircuit 154 within the lens unit 150 to displace the position of thelens 151 via a focus lens driving circuit 156, thereby bringing the lens151 into focus.

A shutter 105 is a focal-plane shutter capable of freely controlling theexposure time of a sensor unit 106 by control of the system control unit132.

The sensor unit 106 includes an image sensor configured of acharge-coupled device (CCD) sensor or a complementarymetal-oxide-semiconductor (CMOS) semiconductor sensor that converts anoptical image into an electric signal, and an analog-to-digital (A/D)converter that converts an analog signal output from the image sensorinto a digital signal and outputs image data. The sensor unit 106 mayinclude an imaging plane phase difference sensor that outputsamount-of-defocus information to the system control unit 132.

A front engine 130 is configured as a single semiconductor integratedcircuit chip (IC chip). The front engine 130 includes one or moreprocessors or circuits. The front engine 130 includes an imageprocessing unit 131 that processes image data acquired from the sensorunit 106, and the system control unit 132 that controls the entiredigital camera 100. The front engine 130 executes a display controlprocess for controlling the display unit 101 and the EVF 108 to displaya live view image on at least one of the display unit 101 and the EVF108 using the image data acquired from the sensor unit 106. “Live view”(live view function) refers to a function for the user to confirm atarget (object) to be captured, the angle of view, and imagingconditions using an image displayed on the display unit 101 and the EVF108. A “live view image” refers to an image displayed on a displaydevice in the live view function.

If the live view function is enabled, the sensor unit 106 continuouslyacquires image data. Based on the image data input from the sensor unit106, the front engine 130 generates display image data to be displayedon the display unit 101 and the EVF 108. Further, the front engine 130controls at least one of the display unit 101 and the EVF 108 to displayan image based on the generated display image data. If the digitalcamera 100 is connected to an external display device (externalapparatus) via a communication unit 109, and the output function to theexternal apparatus is enabled, the live view function may be executedusing the external apparatus (live view output process). In this case, amain engine 140 acquires image data from the front engine 130, generatesdisplay image data, and controls the display unit 101 and the EVF 108.

According to the operation mode of the digital camera 100, the frontengine 130 controls the start of the main engine 140. The front engine130 is a semiconductor integrated circuit at least different from themain engine 140. The front engine 130 will be described in detail below.

A system memory 133 is a non-volatile storage medium that storesprograms and parameters for the system control unit 132 of the frontengine 130 to control the operation of the entirety of the digitalcamera 100. The “programs” as used herein refer to programs forexecuting various flowcharts in the present exemplary embodiment. As thesystem memory 133, for example, a flash memory is used.

A memory 134 is a storage medium in which the image processing unit 131stores the image data to be processed and the processed image data, whenimage data is processed. For example, the memory 134 is a dynamicrandom-access memory (DRAM). Alternatively, parts of the image data tobe processed and the processed image data can also be stored in a partof the system memory 133.

The main engine 140 is configured as a single semiconductor integratedcircuit chip (IC chip) different from the front engine 130. The mainengine 140 includes one or more processors or circuits. The main engine140 includes an image processing unit 141 that processes image dataacquired from the front engine 130, and a control unit 142 that controlsthe functional units of the main engine 140. The main engine 140 alsoincludes a recording/reproducing unit 143 that stores in the recordingmedium 200 the image data processed by the image processing unit 141.The recording/reproducing unit 143 also reads image data from therecording medium 200 and outputs the read image data to the imageprocessing unit 141. If the digital camera 100 operates in thereproduction mode, the image data processed by the image processing unit141 is output to the front engine 130, and an image is displayed on thedisplay unit 101. If an external output function is enabled, the imagedata read by the recording/reproducing unit 143 and processed by theimage processing unit 141 is output to the external apparatus of thedigital camera 100 via the communication unit 109. The main engine 140will be described in detail below.

A system memory 144 is a non-volatile storage medium that storesprograms and parameters for the control unit 142 of the main engine 140to control the functional units of the main engine 140. As the systemmemory 144, for example, a flash memory is used.

A memory 145 is a storage medium in which the image processing unit 141stores the image data to be processed and the processed image data, whenimage data is processed. For example, the memory 145 is amagneto-resistive random access memory (MRAM).

A power supply control unit 107 includes a battery detection circuit, adirect-current-to-direct-current (DC/DC) converter, and a switch circuitfor switching blocks to which current is supplied. The power supplycontrol unit 107 detects the presence or absence of attachment of apower supply 210, the type of the power supply 210, and the remainingamount of power of the power supply 210 (remaining life of battery). Thepower supply control unit 107 controls the DC/DC converter based on thedetection results and an instruction from the system control unit 132 tosupply required power to the components including the front engine 130and the main engine 140 for a required period. The power supply controlunit 107 also supplies power to the recording medium 200 and the lensunit 150. Based on image data acquired by the sensor unit 106, the powersupply control unit 107 displays a live view image on the display unit101 or the EVF 108, and in the image capturing standby state, whereimage data is not recorded in the recording medium 200, the power supplycontrol unit 107 limits power supplied to the main engine 140.

The power supply 210 includes a primary battery such as an alkalinebattery or a lithium battery, a secondary battery such as anickel-cadmium (NiCd) battery, a nickel-metal hydrate (NiMH) battery, ora lithium-ion (Li) battery, or an alternating current (AC) adapter. Thepower supply 210 is attachable to and detachable from the digital camera100.

The EVF 108 displays an image generated by the front engine 130 on adisplay device such as a liquid crystal display (LCD) or an organicelectroluminescent (EL) display. The user can confirm an optical imagedisplayed on the EVF 108 through the eyepiece portion 11.

The communication unit 109 connects to the external apparatus wirelesslyor via a cable for a wired connection, and transmits and receives avideo signal and a sound signal to and from the external apparatus. Forexample, the communication unit 109 converts a signal into a formatcompliant with the High-Definition Multimedia Interface (HDMI®) standardand outputs the signal to the external apparatus. The communication unit109 can also connect to a wireless local area network (LAN) or theInternet. The communication unit 109 can also communicate with theexternal device also using Bluetooth® or Bluetooth® Low Energy. Thecommunication unit 109 can transmit an image (including a live viewimage) captured by the digital camera 100 or an image stored in therecording medium 200. The communication unit 109 can also receive animage or various other pieces of information from the external device.

The operation unit 110 is an operation unit including one or moreoperation members for inputting various operation instructions to thesystem control unit 132. For example, the mode switch 121 is used toswitch the operation mode of the system control unit 132 to any one of astill image capturing mode, a moving image capturing mode, and thereproduction mode. The still image capturing mode includes an auto imagecapturing mode, an auto scene distinction mode, a manual mode, a stoppriority mode (Av mode), a shutter speed priority mode (Tv mode), and aprogram AE mode (P mode). The still image capturing mode includesvarious scene modes in which image capturing settings are made accordingto image capturing scenes, and a custom mode. Using the mode switch 121,the user can directly switch to any one of these modes. Alternatively,using the mode switch 121, the user may once switch to a list screen ofimage capturing modes, then select any one of a plurality of modesdisplayed on the list screen, and switch to the selected mode usinganother operation member. Similarly, the moving image capturing mode mayalso include a plurality of modes. Each image capturing mode includes astate where a live view process for displaying a live view image forimage capturing preparation such as the confirmation of the state of anobject to be captured and the angle of view is executed, and a statewhere an image capturing/recording process for capturing an object andrecording obtained image data in the recording medium 200 is executed.

A first shutter switch 102 a is turned on in an intermediate state of anoperation, i.e., by a so-called half press (image capturing preparationinstruction), on the shutter button 102 provided in the digital camera100 and generates a first shutter switch signal SW1. In response to thereception of the first shutter switch signal SW1, the system controlunit 132 executes an image capturing preparation operation such as an AFprocess, an AE process, an automatic white balance (AWB) process, or apre-flash (EF) process. In other words, the first shutter switch signalSW1 is an image capturing preparation instruction.

A second shutter switch 102 b is turned on by the completion of anoperation, i.e., by a so-called full press (image capturinginstruction), on the shutter button 102 and generates a second shutterswitch signal SW2. Based on the second shutter switch signal SW2, thesystem control unit 132 starts a series of operations of an imagecapturing process from the reading of a signal from the sensor unit 106to the writing of a captured image as an image file to the recordingmedium 200.

In response to the reception of the second shutter switch signal SW2,the system control unit 132 controls the sensor unit 106 to generateimage data. Further, in response to the reception of the second shutterswitch signal SW2, the system control unit 132 controls the power supplycontrol unit 107 to cancel the limitation of power supplied to the mainengine 140, and starts the main engine 140. Then, the main engine 140records, in the recording medium 200, the image data generated by thesensor unit 106 and acquired via the front engine 130. In other words,the second shutter switch signal SW2 is an image capturing instruction.

The recording medium 200 is a recording medium such as a memory card forrecording a captured image and is composed of a semiconductor memory ora magnetic disk.

Image processing of the front engine 130 and the main engine 140 isdescribed.

FIG. 3 is a schematic diagram illustrating functional blocks of theimage processing unit 131 and the system control unit 132 of the frontengine 130 and a data flow in the image capturing mode. The imageprocessing unit 131, the image processing unit 141, and therecording/reproducing unit 143 each include processing units forprocessing image data. Processing executed by each processing unit maybe executed by an electronic circuit, or may be executed by a processorincluded in the image processing unit executing a program.

If the second shutter switch signal SW2 is input from the shutter button102, the system control unit 132 starts an image capturing/recordingprocess. The system control unit 132 controls the shutter 105 and thesensor unit 106 to input an optical image of an object as an imagecapturing target through the lens 151 and form the optical image on thesensor unit 106. The operations of the lens 151 and the sensor unit 106when an image is captured are executed based on parameters acquired inadvance by an image capturing preparation operation executed in responseto the input of the first shutter switch signal SW1. If the parametersare not acquired because the period from the input of the first shutterswitch signal SW1 to the input of the second shutter switch signal SW2is short, the parameters stored in advance in the memory 134 are used.The parameters are determined by the system control unit 132 based onthe results of calculating the evaluation values of the aperture, focus,and camera shake, and object information such as a face recognitionresult.

The sensor unit 106 converts light transmitted through a mosaic colorfilter of red, green, and blue (RGB) arranged on a pixel-by-pixel basisinto electric signals. The number of pixels of an imaging sensor is 4K(8 megapixels or more) or 8K (33 megapixels or more), for example. Theimaging sensor corresponding to 4K includes, for example, 3840horizontal×2160 vertical pixels arranged in a matrix. The imaging sensorcorresponding to 8K includes, for example, 7680 horizontal×4320 verticalpixels arranged in a matrix. FIG. 13 is a diagram illustrating anexample of the color filter arranged in the sensor unit 106 andrepresents the pixel arrangement of an image treated by the digitalcamera 100. As illustrated in FIG. 13, in this structure, red (R), green(G), and blue (B) are arranged on a pixel-by-pixel basis in a mosaic.Then, in every 2×2 (four) pixels, a single red pixel, a single bluepixel, and two green pixels are regularly arranged as a single set. Theelectric signals converted by the sensor unit 106 include red (R), green(G), and blue (B) components. Green (G) can also be treated as two typesof green (G0 and G1) components different depending on positions. Sucharrangement of pixels is generally termed the “Bayer arrangement”. Thesensor unit 106 can output a frame (image data) composed of 4K or 8Kpixels at 120 frames per second.

The electric signals converted by the sensor unit 106 are input to theimage processing unit 131 of the front engine 130.

The image processing unit 131 is an image processing circuit includingone or more processors and circuits that process input image data. Asillustrated in FIG. 3, the image processing unit 131 includes aplurality of functional blocks (processes). Each functional block(process) may be implemented by an individual program or electroniccircuit, or the plurality of functional blocks may be implemented by asingle program or electronic circuit. In the transmission and receptionof data between the functional blocks of the image processing unit 131,the data may be directly transmitted between the functional blocks, or apreprocessing functional block may store the data in the memory 134, anda post-processing functional block may read the data from the memory134.

The image processing unit 131 includes an image processing path used todisplay a live view image or acquire imaging parameters, and an imageprocessing path used for the image capturing/recording process. First, adescription is given of the processing of the image processing unit 131in the image processing path used to display a live view image oracquire imaging parameters.

A pixel rearrangement processing unit 301 rearranges the input electricsignals in a two-dimensional matrix, thereby generating raw image data.In the present exemplary embodiment, the raw image data includesso-called Bayer raw image data output from the sensor unit 106 andhaving monochromatic color information with respect to each pixelcorresponding to the pixel arrangement of the sensor unit 106. In thepresent exemplary embodiment, the raw image data includes image dataobtained by applying compression or correction on partial data to theBayer raw image data output from the sensor unit 106. In other words,image data obtained without applying a so-called “development process”to the Bayer raw image data output from the sensor unit 106 is referredto as the “raw image data”.

A sensor correction processing unit 302 executes on the raw image data acorrection process (sensor correction process) based on thecharacteristics of the sensor acquired in advance. The sensor correctionprocess is, for example, the process of correcting the in-planevariation of the photoelectric conversion efficiency (sensitivity) of aphotoelectric conversion element of the sensor unit 106. The raw imagedata is corrected based on the sensitivity distribution of the sensorunit 106 stored in advance in the system memory 133. The correctionprocess by the sensor correction processing unit 302 also includes arestoration process for restoring a pixel. The restoration processincludes the following process. For the value of a missing pixel or apixel having low reliability in the sensor unit 106, the pixel as arestoration target is interpolated using surrounding pixel values, or apredetermined offset value is subtracted from the value of the pixel asthe restoration target. The restoration process may be changed so as notto perform a part or all of the restoration process at this time, and isperformed at the time of development at the subsequent stage.

During the execution of the LV display process, the raw image datasubjected to the sensor correction process is processed by a reductionprocessing unit 303. During the execution of the imagecapturing/recording process, the raw image data subjected to the sensorcorrection process is processed by a raw noise suppression processingunit 311.

The reduction processing unit 303 reduces the resolution of the rawimage data to efficiently perform a display process and a detectionprocess. The reduction processing unit 303 resizes the raw image datahaving high resolution such as 4K or 8K to a high-definition (HD) size(equivalent to 2 megapixels), for example. The HD size is a size where1920 horizontal×1080 vertical pixels are arranged, for example.Hereinafter, a raw image reduced by the reduction processing unit 303will be referred to as “reduced raw image data”.

The reduced raw image data processed by the reduction processing unit303 is processed by an optical correction processing unit 304 and usedto display an LV image. The reduced raw image data processed by thereduction processing unit 303 is also processed by a detectionprocessing unit 310 and used to acquire the results of calculating theevaluation values of the aperture, focus, and camera shake, and objectinformation such as a face recognition result.

The processes performed by the optical correction processing unit 304 toa display processing unit 309 are described. The series of processes isprocessing for displaying an LV image.

The optical correction processing unit 304 executes a correction process(optical correction process) related to the optical characteristics ofthe lens 151 on the reduced raw image data. The optical correctionprocess is, for example, the process of correcting the influence of areduction in the amount of light in a surrounding area due to theaberration of the lens 151.

A noise suppression processing unit 305 performs the process of reducingnoise in the reduced raw image data. This noise suppression process is aprocess generally termed noise removal or noise reduction (NR). Thenoise suppression processing unit 305 reduces the noise components ofthe reduced raw image data by executing a moving average filter processor a median filter process.

An LV development processing unit 306 executes a development process (LVdevelopment) on the reduced raw image data. The LV development processis a simplified development process in which the circuit load by theprocess and a communication band with the memory 134 are lower than in araw (high image quality) development process. The developed reduced rawimage data is referred to as “display image data”.

An LV correction processing unit 307 executes correction processes suchas distortion correction, an enlargement process, and a reductionprocess on the display image data. Based on the resolution of a displaydevice on which a live view image is to be displayed, the LV correctionprocessing unit 307 executes the enlargement process or the reductionprocess.

An LV effect processing unit 308 executes on the display image data aneffect process for obtaining a predetermined display effect. The effectprocess is, for example, the process of performing color conversion onthe display image data into a sepia tone or monochrome, or processingthe image into a mosaic or a painting-like image.

The display processing unit 309 executes a display process fordisplaying an image on the display unit 101 and the EVF 108 using thedisplay image data. The display processing unit 309 performsinput/output correction, gamma correction, and white balance correctionon the display image data. The display processing unit 309 also performsthe process of combining the display image data with assist informationto be displayed with the image based on the display image data. Theassist information includes at least any of menu icons, icons indicatingimaging parameters, and icons indicating imaging conditions.

The icons indicating imaging parameters are, for example, iconsindicating parameters such as the International Organization forStandardization (ISO) sensitivity, the color temperature, the shutterspeed, and the aperture. The icons indicating imaging conditionsinclude, for example, an icon indicating the number of captured imagesin a single-image capturing mode, a continuous image capturing mode, oran interval image capturing mode, an icon indicating whether the flashcan be used, and an icon indicating a set image capturing mode. Theassist information may be superimposed on the peripheral area of theimage based on the display image data, or may be processed to bedisplayed in an icon display area provided along the outer periphery ofthe image based on display image data. The display processing unit 309outputs the display image data subjected to the above process to thedisplay unit 101 or the EVF 108.

Based on the input display image data, the image is displayed on thedisplay unit 101 and the EVF 108. By the above series of processes, thelive view image is displayed on the display unit 101 and the EVF 108.

The detection processing unit 310 executes, on the reduced raw imagedata, the process of detecting parameters for image capturing control.The detection processing unit 310 calculates the evaluation values ofthe focus state and the exposure state from the reduced raw image data.The calculated evaluation values are stored in the memory 134, and basedon evaluation values read from the memory 134 by the system control unit132, the detection processing unit 310 outputs a control signal to thelens unit 150. The detection processing unit 310 has the function ofdetecting and recognizing object information such as a face or a personin image information. For example, the detection processing unit 310detects a face in a screen represented by the image information, and ifthere is a face in the screen, stores information indicating theposition of the face in the memory 134. The system control unit 132authenticates a particular person based on feature information regardingthe face stored in the memory 134. Display information indicating thecalculated evaluation values and the detection and recognition resultsmay be output to the display processing unit 309 and displayed with thelive view image.

Next, a description is given of the processing of the image processingunit 131 in the image processing path used for the imagecapturing/recording process.

The raw noise suppression processing unit 311 performs the process ofreducing noise in the raw image data processed by the sensor correctionprocessing unit 302. Similar to the noise suppression processing unit305, the raw noise suppression processing unit 311 executes a processgenerally termed noise removal or noise reduction (NR). The raw noisesuppression processing unit 311 reduces the noise components in the rawimage data by, for example, executing a moving average filter process ora median filter process.

A compression processing unit 312 applies a compression process to theraw image data processed by the raw noise suppression processing unit311. The various conventional compression processing techniques can beused as the compression process applied to the raw image data. Thecompression processing unit 312 performs high-efficiency coding on theraw image data using a technique such as wavelet transform,quantization, or entropy encoding (difference encoding). Thehigh-efficiency coding performed by the compression processing unit 312may be irreversible coding or reversible coding. In the presentexemplary embodiment, the high-efficiency coding performed by thecompression processing unit 312 is coding for generating a raw file thatcan be restored as a high image quality file without greatly impairingthe quality of raw image data of the original image even if the raw fileis compressed. The compressed raw image data may be temporarily storedand buffered in the memory 134.

A transmission processing unit 313 transmits the compressed raw imagedata to a reception processing unit 321 of the main engine 140. Sincethe compressed raw image data can be temporarily buffered using theimage processing unit 131 and the memory 134 of the front engine 130,the input speed to the main engine 140, which performs a high imagequality development process at the subsequent stage, can be adjusted.Thus, according to the processing speed of the main engine 140, thetransmission rate (the transmission speed) between the transmissionprocessing unit 313 and the reception processing unit 321 can be madelower than the rate of reading image data from the sensor unit 106.

The main engine 140 executes a high image quality development process onthe raw image data acquired from the front engine 130 and storesrecording image data in the recording medium 200. The image processingunit 141 of the main engine 140 processes the raw image data acquiredfrom the front engine 130, thereby generating recording image data. Therecording/reproducing unit 143 stores the recording image data in therecording medium 200.

The image processing unit 141 is an image processing circuit includingone or more processors and circuits that process input image data. Asillustrated in FIG. 3, the image processing unit 141 includes aplurality of functional blocks (processes). Each functional block(process) may be implemented by an individual program or electroniccircuit, or the plurality of functional blocks may be implemented by asingle program or electronic circuit. In the transmission and receptionof data between the functional blocks of the image processing unit 141,the data may be directly transmitted between the functional blocks, or apreprocessing functional block may store the data in the memory 145, anda post-processing functional block may read the data from the memory145.

The reception processing unit 321 receives the compressed raw image datatransmitted from the transmission processing unit 313 and performsinverse transformation of the compression process applied by thecompression processing unit 312, on the compressed raw image data,thereby decompressing the compressed state.

A raw development processing unit 322 executes a development process onthe raw image data, thereby generating recording image data. The rawdevelopment processing unit 322 performs a debayer process (demosaicprocess), i.e., a color interpolation process, on the raw image data,thereby converting the raw image data into a luminance signal and acolor difference signal or an original color signal. Further, the rawdevelopment processing unit 322 removes noise included in the convertedsignals and corrects optical distortion included in the convertedsignals. In the development process executed by the raw developmentprocessing unit 322, processing with higher accuracy than that in thedevelopment process executed by the LV development processing unit 306is performed. Thus, more resources for the circuits and higher powerconsumption than those in the LV development process are required.

A correction processing unit 323 executes correction processes such asdistortion correction, an enlargement process, a reduction process, anda noise suppression process on the image data subjected to thedevelopment process. In a case where the image capturing/recordingprocess is executed, the correction processing unit 323 executes thedistortion correction and the noise suppression process on the recordingimage data subjected to the development process. In a case where a liveview output process for outputting the image data as a live view imageto the external apparatus via the communication unit 109 is executed,the correction processing unit 323 executes the enlargement process orthe reduction process for outputting the image data to a display device,in addition to the distortion correction and the noise suppressionprocess.

In a case where the live view output process is executed, an effectprocessing unit 324 executes on the image data an effect process forobtaining a predetermined display effect. If the external outputfunction is enabled, the image data processed by the effect processingunit 324 is output to an output processing unit 327.

The output processing unit 327 outputs the image data output from theeffect processing unit 324 to the external apparatus via thecommunication unit 109. In a case where the live view output process isexecuted, the output processing unit 327 performs input/outputcorrection, gamma correction, and white balance correction on the imagedata (LV image data) output from the effect processing unit 324. Theoutput processing unit 327 also performs the process of combining, withthe LV image data, a graphical user interface (GUI) representing assistinformation to be displayed with an image based on the LV image data.The assist information is similar to the information described in thedisplay processing unit 309, and therefore is not described. The outputprocessing unit 327 outputs the processed LV image data to the externalapparatus via the communication unit 109.

In a case where a reproduction output process is executed, the outputprocessing unit 327 performs input/output correction, gamma correction,and white balance correction on the image data (reproduction image data)output from the effect processing unit 324. Further, the outputprocessing unit 327 performs the process of combining, with thereproduction image data, a GUI representing assist information to bedisplayed in the reproduction mode. The output processing unit 327outputs the processed reproduction image data to the external apparatusvia the communication unit 109.

A compression processing unit 325 applies a compression process to therecording image data. The compression processing unit 325 applieshigh-efficiency coding (compression coding) to the recording image data,generates image data having a compressed amount of data, and convertsthe image data into a high image quality developed file. If therecording image data is a still image, a Joint Photographic ExpertsGroup (JPEG) compression process is used as the compression process. Ifthe recording image data is a moving image, a standard coding techniquebased on Moving Picture Experts Group (MPEG)-2, H.264, or H.265 can beused to compress the moving image.

A recording processing unit 326 of the recording/reproducing unit 143stores the compressed recording image data in the recording medium 200.

By the above-described processing, a series of processes in the imagecapturing/recording process is executed. In other words, imageprocessing regarding the display process for displaying an LV imagecompletes using only the front engine 130. On the other hand, the imagecapturing/recording process is performed using both the front engine 130and the main engine 140.

FIG. 4 is a block diagram illustrating a data flow in the reproductionmode in the image processing unit 131 and the system control unit 132 ofthe front engine 130.

If the digital camera 100 is operating in the reproduction mode, boththe front engine 130 and the main engine 140 operate in a normal state.The normal state is the state where each engine can execute imageprocessing. In contrast to the normal state, the state where power islimited (limited state) is an operation state where the powerconsumption is at least lower than in the normal state, and for example,is the state where a part or all of the image processing that can beexecuted in the normal state cannot be executed. Even in the limitedstate, each engine can receive an instruction regarding the start of theengine from outside and execute start control. Thus, the limited statecan also be said to be a standby state.

For example, in the normal state, the front engine 130 can execute adisplay control process for displaying an image on the display unit 101or the EVF 108 based on raw image data input from the sensor unit 106.In the normal state, the front engine 130 can also execute the processof compressing raw image data input from the sensor unit 106 and theprocess of outputting the raw image data to the main engine 140. Thefront engine 130 includes the system control unit 132 that controls theoperation of the digital camera 100. Thus, in a case where the powersupply of the digital camera 100 is on, the front engine 130 basicallydoes not transition to the limited state unless the front engine 130transitions to a sleep mode because an operation is not performed for apredetermined time or more.

In the normal state, the main engine 140 can execute a recording controlprocess for processing compressed raw image data input from the frontengine 130 and storing the raw image data in the recording medium 200.In the normal state, the main engine 140 can also execute a reproductiondisplay control process for reading image data stored in the recordingmedium 200, outputting the image data to the front engine 130, anddisplaying an image on the display unit 101 or the EVF 108 using thefront engine 130. Further, the main engine 140 can execute an outputcontrol process for outputting image data input from the front engine130 to the external apparatus via the communication unit 109. On theother hand, in the limited state, the main engine 140 cannot execute atleast any of the recording control process, the reproduction displaycontrol process, and the output control process.

First, a reading processing unit 401 of the recording/reproducing unit143 reads an image file from the recording medium 200. The image file isan image file subjected to compression coding in advance similarly to arecording process. The read image file is output to the image processingunit 141 of the main engine 140.

A decompression processing unit 402 executes a decompression process forapplying inverse transformation to the compression process to the imagefile, thereby generating image data. A transmission processing unit 403transmits the image data to the front engine 130.

An output processing unit 404 processes the image data output from thedecompression processing unit 402 to generate display image data. Then,the output processing unit 404 outputs the display image data to theexternal apparatus via the communication unit 109. The output processingunit 404 performs input/output correction, gamma correction, and whitebalance correction on the image data output from the decompressionprocessing unit 402. The output processing unit 404 also performs theprocess of combining, with the image data, a GUI representing assistinformation to be displayed with an image based on the image data. Theoutput processing unit 404 can also generate thumbnail images forthumbnail display. The output processing unit 404 outputs the processedimage data to the external apparatus via the communication unit 109. Areception processing unit 411 of the front engine 130 receives the imagedata output from the main engine 140.

A display processing unit 412 processes the received image data togenerate display image data. Then, the display processing unit 412displays an image on the display unit 101 or the EVF 108 based on thedisplay image data.

If the user gives an instruction to perform thumbnail display byoperating the operation unit 110, the system control unit 132 controlsthe main engine 140 to read a plurality of pieces of image data to beused for thumbnail display from the recording medium 200 and output theplurality of pieces of image data to the front engine 130. Then, thesystem control unit 132 controls the display processing unit 412 togenerate thumbnail images and a list display screen based on theacquired plurality of pieces of image data.

Next, control of the entire digital camera 100 is described.

FIGS. 5A and 5B are flowcharts illustrating an operation control flow ofthe digital camera 100. These flowcharts start when the power switch 103of the digital camera 100 is operated from off to on. In FIGS. 5A and5B, reference signs A to D are symbols for connecting reference signswritten in the respective figures. For example, if the determination isYes in step S502 in FIG. 5A, the processing proceeds to step S514 inFIG. 5B via reference sign B.

In step S501, a start process for starting the front engine 130 isexecuted. In response to the reception of a start instruction from thepower switch 103, the power supply control unit 107 supplies power tothe front engine 130. The system control unit 132 of the front engine130 reads a start-up program and a parameter from the system memory 133and executes a start operation. Meanwhile, the power supply control unit107 restricts the supply of power to the main engine 140. Morespecifically, the power supply control unit 107 does not supply powerrequired for the main engine 140 to operate in the normal state.

In step S502, the system control unit 132 determines whether theexternal output function of the digital camera 100 is set to “enabled”.The system control unit 132 reads setting data stored in the systemmemory 133 and determines whether the external output function is set to“enabled”. The external output function is the function of outputting alive view image to the external apparatus via the communication unit109. The settings about the external output function is set in advanceby operating the menu screen. At the time of shipment, the externaloutput function is off. FIG. 6 is a schematic diagram illustrating anexample of a screen for setting the external output (HDMI output)function in the menu screen. The menu screen is displayed on the displayunit 101, and the user operates the operation unit 110 (e.g., cross key114 and SET button 115) to set the external output function to either“enabled” or “disabled”. If the external output function is enabled (Yesin step S502), the processing proceeds to step S514. If the externaloutput function is disabled (No in step S502), the processing proceedsto step S503.

In step S503, the system control unit 132 determines whether theoperation mode of the digital camera 100 is the image capturing mode orthe reproduction mode. The system control unit 132 reads setting datastored in the system memory 133 and confirms the operation mode of thedigital camera 100. Alternatively, based on an operation mode indicatedby the mode switch 121, the system control unit 132 confirms theoperation mode of the digital camera 100. If the operation mode of thedigital camera 100 is set to the image capturing mode (Yes in stepS503), the processing proceeds to step S504. If the operation mode ofthe digital camera 100 is set to the reproduction mode (No in stepS503), the processing proceeds to step S512. In step S512, a startprocess for starting the main engine 140 is performed, and in step S513,a reproduction process for reproducing an image is performed. Thisflowchart is described regarding a case where the image capturing modeis the still image capturing mode. Alternatively, the image capturingmode may be the moving image capturing mode.

Further, in step S503, the system control unit 132 determines whetherthe main engine 140 is in the normal state. If the main engine 140 is inthe normal state, the system control unit 132 executes a stop processfor stopping the main engine 140. This is control for, if the externaloutput function is changed from “enabled” to “disabled” as describedbelow, changing the state of the main engine 140 to the limited state.Details of this process will be described below.

In step S504, the system control unit 132 executes a live view process.The live view process is the process of displaying on the display unit101 a live view image acquired from the sensor unit 106. If the liveview process is executed, an image (live view image) based on image datainput from the sensor unit 106 is displayed on the display unit 101 orthe EVF 108. The user confirms the displayed live view image anddetermines the imaging conditions of an image to be captured (recorded).In other words, it can be said that while the live view process isexecuted, and until an instruction regarding the execution of thecapturing of an image, such as the first shutter switch signal SW1 orthe second shutter switch signal SW2, is input, the digital camera 100is in the image capturing standby state.

FIG. 7 is a flowchart illustrating a flow of the live view process.

In step S701, the system control unit 132 controls the operations of thelens 151 and the sensor unit 106 to acquire (capture) an optical image,and outputs image data. For example, in response to an instructionregarding the zoom or the focus from the user, the system control unit132 outputs an instruction to the lens system control circuit 154 viathe communication terminal 104 to control the focus position of the lens151. The system control unit 132 controls the shutter 105 and the sensorunit 106 to form an optical image on the imaging sensor of the sensorunit 106. Based on information regarding evaluation values and objectinformation obtained by a detection process, the system control unit 132controls the adjustment of the focus to a particular object or thetracking of the particular object.

In step S702, the image processing unit 131 executes a reading processfor reading, from the sensor unit 106, electric signals for generatingimage data. At this time, the reading speed is 1000 megapixels(MP)/second.

In step S703, the pixel rearrangement processing unit 301 rearranges theinput electric signals in a two-dimensional matrix, and generates rawimage data.

In step S704, the sensor correction processing unit 302 executes on theraw image data a correction process based on the characteristics of thesensor acquired in advance.

In step S705, the reduction processing unit 303 applies a reductionprocess to the raw image data to generate reduced raw image data.

In step S706, the detection processing unit 310 executes on the reducedraw image data the process of detecting parameters for image capturingcontrol. The process of step S706 may be executed in parallel withanother process in the LV process.

In step S707, the optical correction processing unit 304 executes acorrection process (optical correction process) related to the opticalcharacteristics of the lens 151 on the reduced raw image data.

In step S708, the noise suppression processing unit 305 executes theprocess of reducing noise, on the reduced raw image data to which theoptical correction process is applied.

In step S709, the LV development processing unit 306 applies adevelopment process (LV development process) to the reduced raw imagedata subjected to the noise suppression process, and generates displayimage data.

In step S710, the LV correction processing unit 307 executes correctionprocesses such as distortion correction, an enlargement process, and areduction process on the display image data.

In step S711, the LV effect processing unit 308 executes on the displayimage data an effect process for obtaining a predetermined displayeffect.

In step S712, the display processing unit 309 executes a display processfor displaying an image on the display unit 101 and the EVF 108 usingthe display image data.

In this way, the flow of the LV process is completed.

The description returns to the flowcharts in FIGS. 5A and 5B. Theprocessing proceeds to step S505.

In step S505, the system control unit 132 determines whether the firstshutter switch signal SW1 is input. More specifically, the systemcontrol unit 132 determines whether the user executes a half pressoperation on the shutter button 102. In other words, the system controlunit 132 determines whether the user inputs an execution instruction toexecute an image capturing preparation operation. If the first shutterswitch signal SW1 is input (Yes in step S505), the processing proceedsto step S506. If the first shutter switch signal SW is not input (No instep S505), the processing proceeds to step S510.

In step S506, the system control unit 132 determines whether the secondshutter switch signal SW2 is input. More specifically, the systemcontrol unit 132 determines whether the user executes a full pressoperation on the shutter button 102. In other words, the system controlunit 132 determines whether the user inputs an execution instruction toexecute an image capturing operation. If the second shutter switchsignal SW2 is input (Yes in step S506), the processing proceeds to stepS507. If the second shutter switch signal SW2 is not input (No in stepS506), the processing proceeds to step S510.

In step S507, the start process for starting the main engine 140 isexecuted. The system control unit 132 controls and instructs the powersupply control unit 107 to start the supply of power to the main engine140. More specifically, the power supply control unit 107 lifts thelimitation of power supplied to the main engine 140. The system controlunit 132 controls the control unit 142 of the main engine 140 to executethe start operation of the main engine 140. Details of the startoperation will be described below.

In step S508, an image capturing/recording process is executed. Theimage capturing/recording process is a series of processes from theapplication of a high image quality development process to raw imagedata acquired (captured) by the sensor unit 106 to the recording of theraw image data in a recording medium determined in advance. FIG. 8 is aflowchart illustrating a flow of the image capturing/recording process.

In step S801, the system control unit 132 controls the operations of thelens 151 and the sensor unit 106 to acquire recording image data. Beforetransitioning to the image capturing/recording process, the systemcontrol unit 132 controls the adjustment of the focus to a particularobject, the white balance, the ISO sensitivity, and the exposure basedon information regarding evaluation values and object informationacquired by the detection processing unit 310 in response to the inputof the first shutter switch signal SW1.

In step S802, the image processing unit 131 executes a reading processfor reading, from the sensor unit 106, electric signals for generatingimage data. The image processing unit 131 reads image data of a singleframe in 5 ms from the sensor unit 106. At this time, the reading speedis 1500 MP/second in pixel terms.

In step S803, the pixel rearrangement processing unit 301 rearranges theinput electric signals in a two-dimensional matrix, to generate rawimage data.

In step S804, the sensor correction processing unit 302 executes on theraw image data a correction process (sensor correction process) based onthe characteristics of the sensor acquired in advance.

In step S805, the raw noise suppression processing unit 311 applies theprocess of reducing noise, to the raw image data processed by the sensorcorrection processing unit 302.

In step S806, the compression processing unit 312 applies a compressionprocess on the raw image data processed by the raw noise suppressionprocessing unit 311. The compression processing unit 312 temporarilystores and buffers the compressed raw image data in the memory 134.

In step S807, the transmission processing unit 313 transmits thecompressed raw image data to the reception processing unit 321 of themain engine 140. The transmission processing unit 313 transmits eachframe in 15 ms to the reception processing unit 321. In this case, thespeed of the front engine 130 transferring a single frame to the mainengine 140 is lower than the speed of the front engine 130 reading asingle frame from the sensor unit 106.

In the image processing (development process) executed by the imageprocessing unit 141 of the main engine 140, processing with higheraccuracy than that in the image processing of the front engine 130 isexecuted. Thus, the amount of data (number of frames) that can beprocessed per unit time by the main engine 140 is smaller than that bythe front engine 130.

The compressed raw image data is buffered, whereby the speed oftransferring image data from the front engine 130 to the main engine 140can be made lower than the speed of reading image data from the sensorunit 106. In this way, image data can be transferred at a data rate atwhich the main engine 140 can process the image data. Accordingly, imageprocessing with high accuracy can be executed without implementing alarge-scale image processing circuit capable of executing the imageprocessing with high accuracy at the reading speed of the sensor unit106. The image processing with high accuracy is image processing withaccurate color reproducibility and correction.

In step S808, the reception processing unit 321 receives the compressedraw image data transmitted from the transmission processing unit 313.

In step S809, the reception processing unit 321 performs inversetransformation to the compression process applied by the compressionprocessing unit 312, on the compressed raw image data, to decompress thecompressed state.

In step S810, the raw development processing unit 322 executes adevelopment process on the raw image data, to generate recording imagedata.

In step S811, the correction processing unit 323 applies distortioncorrection and a noise suppression process to the recording image data.

In step S812, the effect processing unit 324 executes an effect processon the recording image data. The effect process is the process ofapplying an effect determined in advance to the recording image data.The effect process is, for example, a monochrome conversion process orthe process of applying various filters.

In step S813, the compression processing unit 325 applies a compressionprocess to the recording image data.

In step S814, the recording processing unit 326 of therecording/reproducing unit 143 stores the compressed recording imagedata in the recording medium 200.

In step S815, the system control unit 132 determines whether acontinuous image capturing function is enabled. The continuous imagecapturing function can be set by the user operating the menu screen. Ifthe continuous image capturing function is enabled (Yes in step S815),the processing proceeds to step S816. If the continuous image capturingfunction is disabled (No in step S815), the image capturing/recordingprocess ends.

In step S816, the system control unit 132 determines whether the secondshutter switch signal SW2 is input. In other words, the system controlunit 132 determines whether the state where the user strongly pushes inthe shutter button 102 continues. If the second shutter switch signalSW2 is input in step S816 (Yes in step S816), the processing returns tostep S801. In step S801, the image capturing/recording process isexecuted again. If the second shutter switch signal SW2 is not input instep S816 (No in step S816), the image capturing/recording process ends.

In the above flowchart, the main engine 140 records in the recordingmedium 200 the image data obtained by applying the decompression processand the development process to the compressed raw image data output fromthe front engine 130. The main engine 140 can also record in therecording medium 200 the compressed raw image data output from the frontengine 130. In this case, the series of processes from steps S809 toS813 is not executed. The recording/reproducing unit 143 records in therecording medium 200 the compressed raw image data received by thereception processing unit 321.

The description returns to the flowcharts in FIGS. 5A and 5B. Uponending the image capturing/recording process, the processing proceeds tostep S509.

In step S509, the system control unit 132 executes the stop process forstopping the main engine 140. The system control unit 132 controls thecontrol unit 142 of the main engine 140 to execute the stop operation ofthe main engine 140. Then, the system control unit 132 controls andinstructs the power supply control unit 107 to stop the supply of powerto the main engine 140. Details of the stop operation will be describedbelow.

In step S510, the system control unit 132 determines whether an endinstruction to end the operation of the digital camera 100 is input.More specifically, the system control unit 132 determines whether thepower switch 103 is operated from on to off. If an operation is notperformed for a predetermined time, it may be determined that the endinstruction is input (so-called sleep operation). If the end instructionis input (Yes in step S510), the processing proceeds to step S511. Ifthe end instruction is not input (No in step S510), the processingreturns to step S502.

In step S511, the system control unit 132 executes a stop process forstopping the front engine 130 including the system control unit 132. Thesystem control unit 132 also executes a stop process on other componentsof the digital camera 100. Consequently, the operation of the entiredigital camera 100 stops, and the camera control ends.

If it is determined in step S503 that the operation mode is not theimage capturing mode (operation mode is reproduction mode) (No in stepS503), the processing proceeds to step S512.

In step S512, the start process for starting the main engine 140 isexecuted. The start process for starting the main engine 140 is aprocess similar to that of step S507, and therefore is not described indetail.

In step S513, the reproduction process is executed. The reproductionprocess is the process of reading image data stored in the recordingmedium 200 and displaying the image data on the display unit 101. FIG. 9is a flowchart illustrating a flow of the reproduction process.

In step S901, the recording/reproducing unit 143 reads image data fromthe recording medium 200. In the recording medium 200, image datacompressed in a format such as JPEG is stored. The image data to be readby the recording/reproducing unit 143 is determined by the systemcontrol unit 132. The recording/reproducing unit 143 transmits the readcompressed image data to the image processing unit 141.

In step S902, the image processing unit 141 executes the process ofdecompressing the acquired compressed image data. The decompressionprocess is inverse transformation to the compression process applied tothe image data.

In step S903, the image processing unit 141 transmits the decompressedimage data to the image processing unit 131 of the front engine 130.

In step S904, the image processing unit 131 receives the image datatransmitted from the main engine 140.

In step S905, the image processing unit 131 executes display control fordisplaying an image on the display unit 101 or the EVF 108 based on thereceived image data.

In step S906, the system control unit 132 determines whether a changeinstruction to change the display image is input through the operationunit 110. For example, the change of the display image is imageadvancement in response to an operation on the cross key 114, or is achange to thumbnail display in response to an operation on theenlargement button 118. If the change instruction to change the displayimage is input (Yes in step S906), the processing returns to step S901.The system control unit 132 transmits to the recording/reproducing unit143 of the main engine 140 an instruction to read image datacorresponding to the changed display image.

If the change instruction to change the display image is not input (Noin step S906), the processing proceeds to step S907. In step S907, thesystem control unit 132 determines whether an instruction to change themode is input. The instruction to change the mode may be input by theuser operating the mode switch 121, or may be input when the userpresses the shutter button 102. If the instruction to change the mode isnot input (No in step S907), the processing returns to step S906. If theinstruction to change the mode is input (Yes in step S907), thereproduction process ends.

Upon ending the reproduction process, the processing proceeds to stepS509 (stop process for stopping main engine 140).

The above-described processing is the camera control process in a casewhere the external output function of the digital camera 100 isdisabled. Through the above-described processing, if the digital camera100 is operating in the image capturing mode, and until an imagecapturing execution instruction is input, it is possible to limit powersupplied to the main engine 140. In a case where an object is capturedusing the digital camera 100, the user may adjust imaging conditions(angle of view, focus position, and zoom position) while confirmingalive view image. Based on the above flow, during the period when theuser adjusts the imaging conditions while confirming the live viewimage, it is possible to reduce power consumed by the main engine 140.Accordingly, the digital camera 100 can be efficiently used so that thedigital camera 100 that operates using a power supply (a battery) with afinite capacity can be driven for a long time. Such an operation isachieved by enabling the front engine 130 to perform the display processfor displaying a live view image with the configuration of the presentexemplary embodiment.

Next, with reference to FIG. 5B, a description is given of the cameracontrol in a case where the external output function is enabled. Thecamera control in a case where the external output function is enabledis control in a case where it is determined in step S502 in FIG. 5A thatthe external output function is enabled.

If it is determined in step S502 that the external output function isenabled (Yes in step S502), the processing proceeds to step S514. Instep S514, the system control unit 132 determines whether the externalapparatus is connected to the communication unit 109. The communicationunit 109 includes a detection unit that detects a connection. Based on asignal indicating a connection detection result output from thedetection unit, the system control unit 132 determines whether theexternal apparatus is connected to the communication unit 109. Themethod for determining whether the external apparatus is connected tothe communication unit 109 may be another method.

For example, when step S502 is executed, the main engine 140 may betemporarily started up, and the control unit 142 of the main engine 140may determine whether the external apparatus is connected to thecommunication unit 109 via the communication unit 109. Then, thedetermination result of the control unit 142 of the main engine 140 maybe output to the system control unit 132 or a temporary memory (notillustrated), and based on the determination result, the system controlunit 132 may execute a determination process. In this case, uponcompleting the connection detection process, the main engine 140transitions to the limited state. The main engine 140 is temporarilystarted up to detect the external apparatus, thereby eliminating theneed for the system control unit 132 to execute the determinationprocess for determining whether the external apparatus is connected, andcommunication control for communicating with the external apparatus.Therefore, it is possible to reduce the system of the front engine 130.In step S514, the system control unit 132 may determine not only merelywhether the external apparatus is connected to the communication unit109, but also whether the communication unit 109 can communicate withthe external apparatus connected to the communication unit 109 accordingto a predetermined communication protocol.

If it is determined in step S514 that the external apparatus isconnected to the communication unit 109 (Yes in step S514), theprocessing proceeds to step S515. If it is determined in step S514 thatthe external apparatus is not connected to the communication unit 109(No in step S514), the processing proceeds to step S503.

In step S515, the system control unit 132 executes the start process forstarting the main engine 140. The process of step S515 is similar to theprocesses of steps S507 and S512, and therefore is not described.

In step S516, the system control unit 132 determines whether theoperation mode of the digital camera 100 is the image capturing mode orthe reproduction mode. The process of step S516 is similar to that ofstep S503, and therefore is not described. If it is determined in stepS516 that the operation mode of the digital camera 100 is the imagecapturing mode (Yes in step S516), the processing proceeds to step S517.If it is determined in step S516 that the operation mode of the digitalcamera 100 is not the image capturing mode (is the reproduction mode)(No in step S516), the processing proceeds to step S523.

In step S517, the system control unit 132 executes a live view outputprocess. The live view output process is the process of outputting alive view image acquired from the sensor unit 106 to the externalapparatus connected to the digital camera 100 via the communication unit109.

FIG. 10 is a flowchart illustrating a flow of the live view outputprocess. In the live view output process, the main engine 140 performs ahigh image quality development process on image data subjected to asimplified correction process by the front engine 130. A correctionprocess and an effect process are performed on the image data subjectedto the high image quality development process by the main engine 140,and the image data is output to the external apparatus via thecommunication unit 109.

In the flowchart of the live view output process illustrated in FIG. 10,the processes of steps S801 to S810 are similar to the processes withthe same step numbers illustrated in FIG. 8, and therefore are notdescribed.

In step S1001, the correction processing unit 323 applies distortioncorrection, a noise suppression process, and an enlargement/reductionprocess to the image data subjected to the high image qualitydevelopment process by the raw development processing unit 322.

In step S1002, the effect processing unit 324 applies an effect processdetermined in advance to the image data corrected by the correctionprocessing unit 323 to confirm a live view image.

In step S1003, the output processing unit 327 applies to the image datathe process of generating display image data. Then, the outputprocessing unit 327 outputs the image data to the external apparatus viathe communication unit 109. The above-described processing is the liveview output process.

The description returns to the flowcharts in FIGS. 5A and 5B.

In step S518, the system control unit 132 determines whether the firstshutter switch signal SW1 is input. The process of step S518 is similarto the process of step S505, and therefore is not described in detail.If the first shutter switch signal SW1 is input (Yes in step S518), theprocessing proceeds to step S519. If the first shutter switch signal SW1is not input (No in step S518), the processing proceeds to step S521.

In step S519, the system control unit 132 determines whether the secondshutter switch signal SW2 is input. The process of step S519 is similarto the process of step S506, and therefore is not described in detail.If the second shutter switch signal SW2 is input (Yes in step S519), theprocessing proceeds to step S520. If the second shutter switch signalSW2 is not input (No in step S519), the processing proceeds to stepS521.

In step S520, the image capturing/recording process is executed. Theimage capturing/recording process is similar to the process of stepS508, and therefore is not described.

In step S521, the system control unit 132 determines whether an endinstruction to end the operation of the digital camera 100 is input. Theprocess of step S521 is similar to the process of step S510, andtherefore is not described in detail. If the end instruction is input(Yes in step S521), the processing proceeds to step S522.

In step S522, the system control unit 132 executes the stop process forstopping the main engine 140. The process of step S522 is similar tothat of step S509, and therefore is not described in detail. Theprocessing proceeds to step S511, and after the stop process forstopping the front engine 130, the camera control ends.

If the end instruction is not input in step S521 (No in step S521), theprocessing returns to step S502. In step S502, it is determined againwhether the external output function is enabled. Then, the processingbranches to the flow starting from step S503 or the flow starting fromstep S514.

If it is determined in step S516 that the operation mode is not theimage capturing mode (is the reproduction mode)(No in step S516), theprocessing proceeds to step S523. In step S523, the system control unit132 executes a reproduction output process. The reproduction outputprocess is the process of outputting an image based on image data storedin the recording medium 200 to the external apparatus via thecommunication unit 109.

FIG. 11 is a flowchart illustrating a flow of the reproduction outputprocess. In the flowchart of the reproduction output process illustratedin FIG. 11, the processes of steps S901, S902, S906, and S907 aresimilar to the processes with the same step numbers illustrated in FIG.9, and therefore are not described.

In step S1101, the output processing unit 404 executes, on the imagedata processed by the decompression processing unit 402, a displaycontrol process for generating display image data, outputting thedisplay image data to the external apparatus, and displaying an image.Then, the output processing unit 404 outputs the image data to theexternal apparatus via the communication unit 109. The above-describedprocessing is the reproduction output process.

The above-described control is a series of processes in the control in acase where the external output function is enabled. By theabove-described control, if the external output function is enabled, itis possible to output to the external apparatus an image to which thehigh image quality development process is applied by the main engine140. If the external output function is disabled, it is possible toreduce the power consumption of the main engine 140.

If the external output function is enabled, and in the middle of theexecution of the flow from step S514, the external output function maybe set to “disabled” by a user operation. In this case, when theprocessing returns from step S520 to step S502, it is determined in stepS502 that the external output function is disabled, and the processingproceeds to step S503. In such a case, at the time when the processingreturns from step S520 to step S502, the main engine 140 operates in thenormal state. Thus, even though the processes of step S503 andsubsequent steps are executed, the power consumption of the main engine140 remains high during the display of a live view image.

Further, if the external output function is enabled, and in the middleof the execution of the flow from step S514, the connection to theexternal apparatus may be terminated. This corresponds to, for example,a case where the cable connected to the communication unit 109 isremoved. In this case, if the processing proceeds from step S514 to stepS503, and the determination process in step S503 is executed, the mainengine 140 continues to operate in the normal state. In this case,similarly, even though the processes of step S503 and the subsequentsteps are executed, the main engine 140 continues to operate in thenormal state during the display of a live view image, and the powerconsumption increases.

When the determination process in step S503 is executed, the main engine140 may be operating in the normal state. As will be described below, ifthe external output function is enabled, the main engine 140 executes anoutput process for outputting image data to the external apparatus.Thus, the main engine 140 is controlled to be in the normal state. In acase where the external output function shifts from “enabled” to“disabled”, or in a case where the connection to the external apparatusis terminated even if the external output function is enabled, theprocessing proceeds to step S503 with the main engine 140 remaining inthe normal state. Thus, the system control unit 132 determines whetherthe main engine 140 is operating in the normal state in step S503. Ifthe main engine 140 is operating in the normal state, the system controlunit 132 executes the stop process for stopping the main engine 140. Inthis way, even if the main engine 140 is operating in the normal statewhen the determination process in step S503 is executed, it is possibleto limit power to the main engine 140. Thus, even the digital camera 100having the external output function can appropriately control theoperation state of the main engine 140 according to the presence orabsence of external output.

Next, a description is given of details of the start process forstarting the main engine 140 and the stop process for stopping the mainengine 140. The start process for starting the main engine 140 is theprocesses executed in steps S507, S512, and S515 in the flowchartsillustrated in FIGS. 5A and 5B. The stop process for stopping the mainengine 140 is the processes executed in steps S509 and S522 in theflowcharts illustrated in FIGS. 5A and 5B.

FIGS. 12A and 12B are flowcharts illustrating flows of the start processfor starting the main engine 140 and the stop process for stopping themain engine 140. FIG. 12A is the flowchart illustrating the flow of thestart process for starting the main engine 140.

In step S1201, the system control unit 132 outputs to the power supplycontrol unit 107 an instruction to start supply of power to the mainengine 140.

In step S1202, the power supply control unit 107 starts the supply ofpower to the main engine 140.

In step S1203, the control unit 142 of the main engine 140 executes theprocess of reading from the system memory 144 a program and a parameterused by the main engine 140 to start.

In step S1204, the control unit 142 controls the start of the mainengine 140.

By the above series of processes, the main engine 140 transitions fromthe limited state to the normal state.

FIG. 12B is the flowchart illustrating the flow of the stop process forstopping the main engine 140.

In step S1211, the system control unit 132 outputs an operation stopinstruction to the control unit 142 of the main engine 140.

In step S1212, the control unit 142 stores, in the system memory 144,data (start data) such as a parameter required for the next start of themain engine 140. The storage location of the data may be the memory 145.

In step S1213, the control unit 142 executes the process of stopping theoperation of the main engine 140.

In step S1214, the system control unit 132 outputs to the power supplycontrol unit 107 an instruction to stop the supply of power to the mainengine 140.

In step S1215, the power supply control unit 107 stops the supply ofpower to the main engine 140.

By the above-described series of processes, the main engine 140transitions from the normal state to the limited state.

The main engine 140 transitions to the limited state, whereby it ispossible to reduce the power consumption of the digital camera 100.

Through the above-described flows, the system control unit 132 performscontrol to restrict the supply of power to the main engine 140 in theimage capturing standby state, and in response to an image capturinginstruction (SW2) to record image data in a recording medium, lift therestriction of the supply of power to the main engine 140. However, atrigger for lifting the restriction of the supply of power to the mainengine 140 is not limited to the image capturing instruction (SW2). Forexample, in response to an image capturing preparation instruction(SW1), which is a preliminary operation before an image is captured, therestriction of the supply of power to the main engine 140 may be lifted.With this operation, it is possible to complete the start of the mainengine 140 before the second shutter switch signal SW2 is input. Thus,it is possible to acquire and record an image at the timing when thesecond shutter switch signal SW2 is input. In other words, the firstshutter switch signal SW1, which is an image capturing preparationinstruction, can also be said to be an instruction regarding an imagecapturing operation for recording image data acquired from the sensorunit 106 in a recording medium.

With the digital camera 100, according to the operation mode or theprocess to be executed, the operation states of the front engine 130 andthe main engine 140 are controlled. Accordingly, it is possible toefficiently control the operations of a plurality of engines accordingto the state of the digital camera 100 and reduce the power consumptionof the digital camera 100. Further, it is possible to generate andoutput image data having appropriate quality according to the operationmode.

According to embodiments of the present disclosure, in an imagingapparatus that uses a plurality of image processing circuits to processimage data acquired from a sensor, it is possible to control the supplyof power to each image processing circuit according to the operation ofthe imaging apparatus and achieve the efficient operation of the imagingapparatus.

While the present disclosure includes exemplary embodiments, it is to beunderstood that the disclosure is not limited to the disclosed exemplaryembodiments. The scope of the following claims is to be accorded thebroadest interpretation so as to encompass all such modifications andequivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2019-152245, filed Aug. 22, 2019, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An imaging apparatus comprising: an imagingsensor; a display; a first integrated circuit (IC) chip that receivesimage data from the imaging sensor; a second IC chip that receives imagedata from the first IC chip; and a power control circuit, wherein thefirst IC chip applies a predetermined process for displaying andrecording to the image data from the imaging sensor and outputs theimage data to which the predetermined process is applied to the displayin an image capturing standby state and an image capturing state of theimaging apparatus, wherein the first IC chip outputs the image data towhich the predetermined process is applied to the second IC chip in theimage capturing state, wherein the second IC chip receives the imagedata to which the predetermined process is applied from the first ICchip and performs a process for recording the image data received fromthe first IC chip on a recording medium, wherein the recording of theimage data on the recording medium is not performed in the imagecapturing standby state and the recording of the image data on therecording medium is performed in the image capturing state, and whereinthe power control circuit supplies power to the first IC chip andrestricts supply of power to the second IC chip in the image capturingstandby state, and, upon an instruction for recording image data, liftsthe restriction of the supply of power to the second IC chip.
 2. Theimaging apparatus according to claim 1, wherein the first IC chipdisplays, in the image capturing standby state, a live view image on thedisplay nit based on the image data acquired from the imaging sensor. 3.The imaging apparatus according to claim 1, wherein the first IC chipdisplays, in the image capturing standby state, an image on the displayusing image data obtained by applying a first development process to theimage data to which the predetermined process is applied.
 4. The imagingapparatus according to claim 3, wherein the first IC chip, upon theinstruction, transmits the image data to which the predetermined processis applied to the second IC chip without applying the first developmentprocess to the image data to which the predetermined process is applied,and wherein the second IC chip records, on the recording medium, imagedata obtained by applying a second development process to the image datareceived from the first IC chip.
 5. The imaging apparatus according toclaim 1, further comprising a communication circuit, wherein the secondIC chip outputs image data to an external apparatus via thecommunication circuit.
 6. The imaging apparatus according to claim 5,wherein the power control circuit does not restrict, in a case where afunction for outputting image data to the external apparatus is enabled,the supply of power to the second IC chip even in the image capturingstandby state.
 7. The imaging apparatus according to claim 1, whereinthe first IC chip applies a noise reduction process to the image data towhich the predetermined process is applied, applies a simplifieddevelopment process to the image data to which the noise reductionprocess is applied, and displays an image on the display based on theimage data obtained by the simplified development process, and whereinthe second IC chip applies a development process for generating imagedata having a higher image quality than that in the simplifieddevelopment process, to the image data received from the first IC chip,and records on the recording medium the image data obtained by thedevelopment process.
 8. The imaging apparatus according to claim 1,wherein upon receiving a power-on instruction to start the imagingapparatus, the power control circuit supplies power with which the firstIC chip can execute a start operation, and supplies limited power to thesecond IC chip.
 9. The imaging apparatus according to claim 1, whereinthe power control circuit restricts the supply of power to the second ICchip in accordance with completion of the recording of the image data onthe recording medium in the image capturing state.
 10. The imagingapparatus according to claim 1, further comprising: a first memory forthe first IC chip; and a second memory for the second IC chip.
 11. Theimaging apparatus according to claim 1, wherein the instruction forrecording image data is issued in accordance with an operation of ashutter switch by a user.
 12. The imaging apparatus according to claim11, wherein the operation of the shutter switch includes a first switchoperation for an instruction of an image capture preparation includingan auto-focus process and an auto-exposure process and a second switchoperation, which is executed after the first switch operation, for aninstruction of capturing and recording an image, and wherein the powercontrol circuit does not lift the restriction of the supply of power tothe second IC chip even though the first switch operation is executedand lifts the restriction of the supply of power to the second IC chipin accordance with the second switch operation.
 13. The imagingapparatus according to claim 1, wherein the power control circuitsupplies power necessary for a normal operation of the first IC chip tothe first IC chip in the image capturing standby state and the imagecapturing state, and supplies power necessary for a normal operation ofthe second IC chip to the second IC chip in accordance with the liftingof the restriction of the supply of power to the second IC chip.
 14. Acontrolling method of controlling an imaging apparatus having an imagingsensor, a display, a first integrated circuit (IC) chip that receivesimage data from the imaging sensor, and a second IC chip that receivesimage data from the first IC chip, the method comprising: controllingthe first IC chip to apply a predetermined process for displaying andrecording to the image data from the imaging sensor and output the imagedata to which the predetermined process is applied to the display in animage capturing standby state and an image capturing state of theimaging apparatus; controlling the first IC chip to output the imagedata to which the predetermined process is applied to the second IC chipin the image capturing state; controlling the second IC chip to receivethe image data to which the predetermined process is applied from thefirst IC chip and perform a process for recording the image datareceived from the first IC chip on a recording medium, wherein therecording of the image data on the recording medium is not performed inthe image capturing standby state and the recording of the image data onthe recording medium is performed in the image capturing state;supplying power to the first IC chip and restricting supply of power tothe second IC chip in the image capturing standby state; and lifting therestriction of the supply of power to the second IC chip upon aninstruction for recording image data.